The impact of Quality Indicators on the rating of Multi-objective Evolutionary Algorithms

Comparing the results of single objective optimizers is an easy task in comparison to multi-objective optimizers for which the result is usually an approximation of the Pareto optimal front. These approximation sets must first be evaluated. One of the most popular methods for evaluation is the use of quality indicators, for which the result is a real valued number that reflects a certain aspect of quality. Evaluating and comparing multi-objective optimizers is an important issue. It has been empirically proven that chess ranking can be successfully applied to ranking and comparing single objective evolutionary algorithms. In this paper, the method was adapted to multi-objective evolutionary algorithms (MOEAs). The comparison of several different quality indicators in the chess rating system was conducted in order to get a better insight on their characteristics and how they affect the ranking of MOEAs. Although it is expected that quality indicators with the same optimization goals would yield a similar ranking of MOEAs, it has been shown that results can be contradictory.